Storage servers can store data redundantly, e.g., across multiple data storage devices. Storage servers may employ various forms of data storage devices, such as hard disk drives, solid state drives, flash drives, or tape devices for example. The data storage devices are typically implemented as one or more storage volumes that comprise a subset of the data storage devices, in which the volumes define an overall logical arrangement of storage space. For example, a storage server can serve a large number of discrete volumes each generally associated with its own file system.
To improve performance, storage servers can temporarily store various storage modify operations and associated data received from client devices in a region of system memory. Storage operations generally can be classified into storage operations that modify the state of the storage system and those that only read the state of the storage system. By storing the storage modify operations and data they receive in system memory, the storage servers can immediately return an acknowledgement message to the client devices rather than wait for slower data storage devices to actually store the data prior to the acknowledgement being sent. However, system memory can be erased before the data is stored to data storage devices, e.g., in the event of a power (or other) failure.
To eliminate the possibility of data loss in such circumstances, storage servers may also store the storage operations and associated data in non-volatile random access memory (NVRAM), e.g., in a log stored in the NVRAM. By initially storing the storage operations in the log, the storage server can immediately return an acknowledgment to the client devices rather than wait for the operation to complete on one or more data storage devices.
Moreover, in the event of failure of the storage server, the storage operations can be replayed, thereby preventing loss of data. The NVRAM can have various associated circuitry to prevent data loss, e.g., battery backup, flash-type memory, etc. By logging storage operations as “journal” entries in the log, a storage server can conform with data storage protocols that require the storage server only acknowledge storage operations after writing data to persistent storage. Journaled storage operations can include external operations received from client devices (e.g., create file, write data, delete data, etc.) as well as internal operations generated in order to manage a hosted file system, for example.
The log can accumulate storage operations until a consistency point is triggered. Consistency points can be triggered at various time intervals (e.g., fixed time intervals), or when other events arise, e.g., the NVRAM is almost fully filled. At each consistency point, data is transferred from the storage server system memory (e.g., the NVRAM) to underlying data storage volumes on data storage devices as a result of executing the storage operations in the log, and the system memory is cleared of the transferred data upon successful transfer. If the storage server's operations are interrupted unexpectedly, e.g., because of power failure or other subsystem problem, its operating system or file system can recover by using information stored in the log between the time of the last consistency point and the unexpected interruption, e.g., by using a replay operation.
Storage servers incur runtime costs in order to perform the journaling operations and populate and maintain the log, including with respect to processor utilization and NVRAM utilization. Additionally, replay costs are incurred by storage servers when the storage servers are recovering from a crash or other failure. In particular, the replay costs include replaying the storage operations in the log in the NVRAM. These costs impact performance of the storage servers, as well as the duration the storage servers are unavailable to service storage operations during a recovery.